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3D . GRATING OPTICAL SENSOR HAVING A DIFFUSION GLASS FOR 
^giiDLiCXIN£_ CHROMATOMETRY W ITH^ ^COLOR, CONSTANCY 
PERFORMANCE 

The invention relates to a grating optical 
sensor comprising the features of the preamble of 
claim 1 . 

Such a sensor is disclosed in WO 97/22 849. It 
is provided for accurately determining spatial and/or 
temporal spacings in focused image sequences of a 
lens/pupil system and/or determining spatial and/or 
temporal objecf parameters in real time such as, for 
example, sp^^^d or depth. A 3D grating has also already 
been usedr to carry out model calculations relating to 
the ojpverted retina of the human eye and to relate them 
l^fsubi ective phenomena known from human vision. In the 
preferred form, the 3D grating has a hexagonal 
structure. Other structures with centrosymmetrical 
diffraction patterns are, however, likewise possible. 

Since the investigations of O. Lummer and the 
industrial development of daylight-like luminaires, it 
has been realized that there is an as yet unexplained 
resonance between sunlight and human vision. This has 
resulted in all the previous recommendations for 
approximating the spectrum of artificial light sources 
to the sunlight spectrum. In particular, in the case of 
color perception in phototopic day vision, there occur 
in the event of a change of illuminations having a 
different spectral composition of the radiation 
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displacements of the color values which are compensated 
adaptively in human vision after a relatively short or, 
in part, relatively long time by means of approximate 
color constancy performances of the eye. The v. Kreis 
model, which attributes the adaptivity to the visual 
pigments of the retina, presently serves as an 
incomplete explanatory model for this. In addition, 
there are even more incomplete cortical explanatory 
models from other authors. 

On the other hand, it has been documented many 
times that the phototopic seeing process cannot be 
characterized solely by the spectral light sensitivity 
of the individual cones. The very much more complex 
mode of operation of the visual sense requires 
knowledge of the luminance distribution in the entire 
visual field for the purpose of judging many visual 
tasks. Human vision is not based on the stimulus/ 
reaction response of individual pixels. It takes 
account of the relative values over the entire field of 
view. In addition to chromatic adaptation effects, 
scattering of light at ocular media influences the 
extent of the achromatic axis (black-gray-white axis) 
centering the color space. It is therefore an illusion 
to believe that spectral photometers will be the ideal 
instruments of future chromatometry and color 
determinations, even if they are designed on the 
detection of overlapping RGB values. Likewise 
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incomplete is a chromatometric technique which 
respectively dispenses with determining the triad of 
brightness/hue/saturation simultaneously and with 
reference to a entire field of view. 

There is thus a growing need to have available 
in the future color sensors which can measure color 
values with reference to the spectral sensitivity 
curves of human vision, and ensure, given adaptation to 
artificial illuminations, an approximate color 
constancy corresponding to human vision. It is the 
object of the invention to create such a sensor. 

This object is achieved in the case of a 
grating optical sensor of the type mentioned at the 
beginning by means of the characterizing features of 
the device claim 1 and of the method claim 15. 
Advantageous developments follow from the features of 
the associated subclaims. 

The invention proceeds from the finding that it 
is possible, by inserting a diffractive multilayer (3D) 
grating into the image plane of an imaging lens/pupil 
system in the near field downstream of the grating 
(Fresnel/Talbot space; Fourier space or reciprocal 
grating), to make available three chromatic diffraction 
orders (RGB triple) with in each case six discrete 
interference maxima on mutually concentric circles, 
such as are described in the case of a hexagonal 


wo 01/86242 - 4 - PCT/DEOl/01568 

grating structure by means of the v. Laue equation 
known from crystal optics. 

The V. Laue equation for diffractive space 
lattices requires for the production of constructive 
interference maxima the simultaneous satisfaction of 
the three phase conditions in the equation 1 1 1 

(cosa-cosa**) = hi X/gx 

(cosp-cosp°) = h2 X/qy HI 

(cosy-cosy'*) = h3 X/gz 

(hlh2h3 = triple of integral diffraction orders n; a**, 
p^'^Y** = aperture angle of the light cone incident in the 
3D grating, relative to x, y, z; a, y = angle of the 
diffraction orders relative to x, y, z; X- = wavelength; 
and gx, gy, gz = grating constant in the x-, y-, z- 
axial direction) . Assuming a hexagonal packing of the 
optically diffracting elements and grating constant 
dimensions in ym of gx = 2>.lll, gy = 4?i111/a/3, gz = 
4A.111, in equation |2| A^lll constitutes the wavelength 
diffracted with maximum transmission into the 111 
diffraction order. 


2(^cosa^ +— cosp^ +— ayy^) 
gx^ gy gz 


In the case of perpendicular incidence of light 
(a**=p°=90**, y^O"") a triple of chromatic diffraction 
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orders results in the visible spectral region 
(380-780 nm) 

X.111 (longest wavelength) RED 
A.123 (average wavelength) GREEN 
X122 (shorter wavelength) BLUE 

The spectral transmission curves, which are 
centered relative to one of these A.max in each case, 
have a Gaussian shape and are determined at their half 
width by the number of the surface gratings in the z 
direction that are present in the 3D grating. In the 
event of incidence of white light, that is to say light 
of identical energy in all spectral components, 
[lacuna] the grating inserted into the image plane of 
the imaging system, given the selection of Xlll = 
559 nm, the result is the trichromatism of the 
diffraction orders at 
Xlll RED = 559 nm 
A,123 GREEN = 537 nm 
X122 BLUE = 447 nm 

There is thus a trichromatic tuning of the 3D grating, 
which is based on the resonant setting of the grating 
constants gx and gz to an integral Xlll, and in which a 
trichromatic equilibrium of the brightness values 
(Patterson amplitudes^ weights) is produced in the RGB 
diffraction orders. 

In the case of adaptive chromatic retuning of 
the 3D grating to an illumination other than a white 
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one, the relation of the RGB kmax (1 : 0.96 : 0.8 or 
25 ; 24 : 20) is always maintained. A.111 as the 
resonant wavelength determining the triple shifts to 
shorter A.111 wavelengths in the event of change to a 
blue illumination, and to longer Xlll wavelengths in 
the event of change to a red illumination. The adaptive 
shift ends with the complete adaptation to the new 
illumination, that is to say with the resonant finding 
of a new RGB equilibrium, of the trichromatically 
additive white standard, which recenters the color 
space. The actual resonance factor is the phase 
velocity nvX = c (n = refractive index of the medium, v 
= frequency of the light, c = speed of the light) . 

The following new configuration of the sensor 
design forms the basis for the color constancy 
performance of the 3D grating optical sensor in the 
case of adaptation to variable illuminations. 

A diffusion glass or one or more light 
diffusing gratings are incorporated into the pupil 
plane (aperture space) of the imaging optical system. 
Their function is to be seen in that they scatter 
diffusely as incoherent background into the image plane 
information likewise present at each location in the 
pupil, spatially the sum of the spectral intensities 
and local frequency values which are irradiated into 
the pupil by all objects in the object space and 
contribute to optical imaging. As a result, each local 
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image location is supported by the global information 
on the entire field of view, against which each local 
pixel must stand out by being differentiated from it, 
specifically in brightness, hue, saturation etc. 
However, each item of local information thereby remains 
relativized in terms of the global background of the 
entire field of view. 

All the lenses, diffusion glasses or gratings are 
designed such that they are transparent exclusively for 
electromagnetic radiation in the visible spectrum (380- 
780 nm) and therefore these delimit an octave of the 
wavelengths or frequencies with, definite absorption 
edges. This boundary condition is important because 
thereby spectral brightness values which could come 
about through variation in the illumination are cut off 
at these absorption edges. 

In the near field downstream of the diffractive 3D 
grating, the RGB interference maxima (3x6 concentric 
maxima) assigned to a local pixel are interconnected 
via photoreceivers set in a constant fashion in terms 
of their spectral sensitivity to white sunlight (of 
identical energy in all spectral components) in such a 
way that a local RGB sum can be formed as a 
trichromatically additive brightness value by means of 
an appropriate evaluation. It is possible thereby to 
differentiate RGB equilibriums and disequilibriums. RGB 
equilibriums correspond in the object space to visible 
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colorless surfaces or illuminations (black-gray-white 
objects) . If illuminations are not visible, but can be 
inferred only via colorless objects or surfaces, they 
are represented in terms of their spectral 
characteristic by gray or white surfaces, what are 
termed mirrors of the illuminations. The image location 
with RGB equilibrium, which achieves the greatest 
aggregate brightness, supplies what is termed the white 
standard, and thereby defines the tip of the achromatic 
axis centering the color space. Alternatively, the 
image location whose RGB values most closely 
approximate an equilibrium takes over this provision of 
a white standard. This explains that the white standard 
can be displaced in the trichromatic space. 

The design of a diffractive 3D grating optical 
sensor which provides trichromatic RGB values in three 
diffraction orders ensures color constancy when there 
is ensured together with the sudden or gradual change 
in the illumination in the object space a resonant 
mechanism, that is to say one that is adaptive to the 
spectral composition of the illuminations in the 3D 
grating, which corresponds to a chromatic tuning of the 
3D grating. In the case of a white illumination, that 
is to say an equal-energy spectral composition, 
corresponding to average sunlight, of the visible 
light, three grating constants in the xyz-axial 
direction are tuned to the RED wavelength (559 nm) , 
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[illegible] standing wave formation in the x- and z- 
axial directions, that is to say resonance in the 3D 
grating. Identical values, that is to say RGB 
equilibriums, result thereby under the three Gaussian 
spectral photopic curves of the photoreceptors (cones 
in human vision) . The white standard is determined via 
the RGB sum values of the three Gaussian curves, which 
are centered relative to the wavelengths 559 nm 
RED/537 nm GREEN/447 nm BLUE. 

After a sudden or gradual change in the 
illumination, a chromatically triggered reconstruction 
of the grating constants takes place in the diffractive 
3D grating. In the case of a displacement of 
illumination to the longer wavelength region of the 
spectrum, the white standard in the 3D grating, still 
tuned to 559 nm RED, suddenly breaks down. If the 
adaptive mechanism of the shifting of the white 
standard then acts in the direction of the changed 
illumination, the grating achieves a new RGB 
equilibrium in the case of a chromatic tuning to 728 nm 
RED, for example. The trichromatically additive color 
space is thereby centered again relative to an 
achromatic axis, and the colors are correct again, 
being experienced as correct. 

If, by contrast, the illumination is displaced 
toward the shorter wavelength spectrum, the grating 
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achieves a new RGB equilibrium, for example in the case 
of a chromatic tuning to 513 nm RED. 

The adaptive process, which leads as a result 
to a trichromatic restandardization of the color space 
in a changed white standard, can be described by the 
already explained v. Laue equation of crystal optics. 
The actual resonance factor is the phase velocity in 
the medium. Spectral triggering of the grating constant 
dimensions corresponds to the coefficient of thermal 
expansion for the RED wavelength in the RGB triple. It 
is possible by means of dosed IR, that is to say 
thermal irradiation in the 3D grating, or by varying 
the internal pressure in the 3D grating to vary the 
grating constant dimensions correspondingly. The sensor 
according to the invention can thereby ensure the color 
constancy properties of the human vision system. 

A color constancy sensor which is represented 
technically in the form of a 3D grating or 
preprocessing filter that can resonate with the 
centroid of the spectral component of a light source or 
illumination is very important for all applications in 
which the color of fabrics and materials must be 
detected, differentiated and classified by the color 
perception forming the basis of the laws of human 
vision. This also holds for the relevant judgment of 
properties of visible objects that are associated with 
the hue characteristics, whether this be in general 
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image processing, whether with automatic viewers in 
robotics or autonomously driving vehicles, indeed even 
in the case of sensors for the blind. At the same time, 
such a sensor is able to render color perceptions under 
artificial light sources predictab-le -and measurable. 
Since such a 3D grating transforms the physical 
parameters (intensity and wavelength) into the 
psychological triad of brightness, hue and saturation, 
it is also possible to use a sensor to calculate 
brightness and saturation values of object surfaces. 

An exemplary embodiment of the grating optical 
sensor according to the invention is illustrated 
diagrammatically in the drawing and described with the 
aid of the figures, in which: 

Figure 1 shows the design of the sensor. 

Figure la shows the plan view of a centrosymmetrically 

trichromatic diffraction pattern. 
Figure 2 shows the adaptation to a white illumination. 
Figure 3 shows the adaptation to a red illumination. 
Figure 4 shows the cycle of the operation of adapting 

to a red illumination. 
Figure 5 shows the adaptation to a blue illumination, 

and 

Figure 6 shows the cycle of the operation of adapting 
to a blue illumination. 
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The grating optical sensor illustrated 
schematically in figure 1 includes as lens/pupil system 
an imaging lens 1. The latter projects a visible object 
2, which is illuminated by a radiation source 3 
emitting white light, from the object space onto a 
diffractive 3D grating as modulator 4 with the grating 
constants gx, gy, gz in the image plane 5. Three 
chromatic RGB diffraction orders in the diffraction 
pattern 6 with six concentric maxima (Patterson 
weights) each result in a known way for each imaged 
S object 2 in the visible spectrum owing to diffraction 

R in the hexagonal 3D grating optical modulator 4, 

2 resonance between Xlll and the grating constants as 

fl well as interference in the near field downstream of 

the modulator 4. These are illustrated once again in 
P plan view in figure la for an object 2 situated on the 

optical axis 7 of the sensor. In this case, the red (R) 
diffraction order is situated on the inner ring, the 
blue (B) on the middle ring, and the green (G) on the 
outer ring. 

Each diffraction order is assigned a 
photoelectric receiver 8. All the receivers 8 are set 
to the same spectral sensitivity for a radiation source 
3 emitting white sunlight. 

Objects situated outside the optical axis 7 
supply identical diffraction patterns 6, which can also 
be interleaved. The resolution of the image depends on 


O 

5 
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the grating constants of the 3D grating. Each 
diffraction pattern is assigned a specific object. 

A diffusion glass 9 is inserted into the pupil 
plane of the lens 1 or a plane conjugate thereto. This 
diffusion glass can advantageously have a diffracting 
grating structure. Since imaging beams from all the 
objects in the object space run through each location 
of the pupil, image information from the entire object 
space is distributed simultaneously over the image 
plane via each diffusion center outward from the pupil 
plane- Consequently, information on the overall image 
is superimposed on each local image of an object. The 
diffusion characteristic of the diffusion glass 9 is to 
be selected such that diffusion takes place as 
uniformly as possible over the entire image field, and 
an image of the local object on the background produced 
by diffusion is maintained. 

The spectral transmission of the lens 1, the 
diffusion glass 9 and the modulator 4 are limited to 
the visible region of electromagnetic radiation, in 
particular to the wavelength region of 380-780 nm. 

All the receivers 8, assigned to the same 
diffraction order R, G, B, of a diffraction pattern 6 
are interconnected for the purpose of forming a local 
chromatically additive brightness value 10. The local 
trichromat ically additive brightness values 11 are 
additionally produced therefrom in a downstream summer. 
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The 


local 


chromatically 


additive 


and 



3 y 


trichromatically additive brightness values 10^ 11 of 
the diffraction pattern 5, and the corresponding 
brightness values 10', 11' of other diffraction 
patterns are fed to a comparison arrangement 12 for the 
purpose of determining the diffraction pattern with the 
best agreement between the chromatically additive 
brightness values 10, 10' and, simultaneously, a 
maximum trichromatically additive brightness value 11, 
11'. The corresponding brightness values of the 
selected diffraction pattern are led to a white 
standard forming unit 13 for the purpose of producing a 
white standard value. The agreement between the three 
chromatically additive brightness values means that a 
colorless object detail is involved. The magnitude of 
the trichromatically additive brightness value 
specifies an evaluation on the black-gray-white scale. 

The chromatically additive brightness values 
10, 10' of the individual diffraction patterns can also 
be fed to a color value forming unit 14. The sum of the 
three different chromatically additive brightness 
values that are referred in each case to the white 
standard signal, for example multiplied by the 
reciprocal of the white standard, forms the output 
signal for the local color value. If there is no 
measured white standard value available, the color 
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evaluation can also be carried out by forming the ratio 
with a fictional white standard value. 

The process of white standard formation can be 
checked permanently or in periodic sequence in a 
variation signaling unit 15. A variation in the 
illumination leads either only to a change in the 
trichromatically additive brightness value 11^ 11' of 
the diffraction patterns selected for the white 
standard, while an RGB equilibrium is maintained. The 
white standard value is displaced in this case only on 
the achromatic axis, centering the color space, for 
black-gray-white objects. 

If the variation in the illumination also 
leads, however, to an RGB disequilibrium in the 
diffraction pattern determining the white standard, the 
cause resides in a variation in the spectral 
composition of the illumination. The variation 
signaling unit 15 establishes such a variation and 
controls an adapter 16 which is assigned a thermal 
radiation source 17. Thermal radiation on the modulator 
4 causes variation in its grating constant as a 
function of the coefficient of thermal expansion until 
the white standard forming unit 13 displays a new white 
standard value. This adaptation process corresponds to 
an inclination of the achromatic axis centering the 
color space. 
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Figure 2 firstly illustrates the adaptation to 
a white illumination with spectral components of 
approximately the same energy. The intensity of 
emission is illustrated in the upper diagram as a 
function of wavelength. A diffractive hexagonal 3D 
grating in this case supplies three diffraction orders 
whose Gaussian spectral transmission curves are 
centered in relation to Xlll = 559 nm (RED), 
A.123 = 537 nm (GREEN) and X.122 = 447 nm- (BLUE). This 
corresponds to the sensitivity of the cones in human 
day vision. The Gaussian curves illustrated in the 
lower diagram can be described by a"^exp(-x^), with 
X = (Xhlh2h3 - X) /n and a = 0.92 at n = 55 for HIRED, 
a = 0.88 at n = 53 for 123GREEN, and a = 0.56 at n = 34 
for 122BLUE. The achromatic, that is to say gray to 
white, objects reproduce the spectral properties of the 
illumination in the object space to the extent that 
this itself is invisible. The product of the spectral 
intensities and spectral Gaussian curves produce 
identical brightness aggregate values of 33% each in 
the three diffraction orders. Their RGB equilibrium 
supplies the white standard, which centers the 
trichromiatic color space. In the table below, the 
values of the spectral brightness distribution are 
summarized in accordance with the Gaussian curves 
assigned to the diffraction orders, for the case of a 
white illumination . 
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Figure 3 and figure 4 illustrate the adaptation 
to an illumination displaced to the red. The intensity 
of emission is illustrated once again in the upper 
image as a function of wavelength, and that of the 
associated Gaussian spectral transmission curves is 
illustrated in the lower image of figure 3. The 3D 
grating optical adaptation to a red illumination leads 
via a chromatic tuning of the three grating constants 
at Xlll = 728 nm RED, X123 = 699 nm GREEN, 

X122 = 582 nm BLUE to a new trichromatic RGB 
equilibrium position which is displaced to the longer 
wavelength end of the spectrum and forms the new white 
standard. The product of variable spectral energy 
distribution in the illuminating light and a constant 
triple of the Gaussian curves results in the new 
distributions of the spectral brightness values in the 
RGB diffraction orders, as they are summarized in the 
following table. 
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The resonant adaptation operation illustrated 
in figure 4 starts with a disequilibrium in the RGB 
diffraction orders at 43% R, 39% G, 18% B, which was 
initiated by the sudden change from the white to the 
red illumination. Gradually progressive 3D grating 
optical resonance with longer A.111 wavelengths finally 
leads to the chromatic grating constant tuning at 
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kill = 728 nm RED, and thus to the new RGB equilibrium 
with 33% R, 35% G and 32% B. 

Figure 5 and figure 6 show in a similar way the 
3D grating optical adaptation to a blue illumination 
via a chromatic tuning of the three grating constants 
at Xlll = 513 nm RED, X123 = 492 nm GREEN, 

A,122 = 410 nm BLUE to a new trichromatic RGB 
equilibrium position, that is displaced to the short- 
wave end of the spectrum and forms the new white 
standard. The product of variable spectral energy 
distribution in the illuminating light and a constant 
triple of the Gaussian curves results in the new 
distributions of the spectral brightness values in the 
RGB diffraction orders, as they are summarized in the 
following table . 
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The resonant adaptation operation illustrated 
in figure 6 starts with a disequilibrium in the RGB 
diffraction orders at 24% R, 28% G, 48% B, which was 
initiated by the sudden change from the white to the 
blue illumination. Gradually progressive 3D grating 
optical resonance with shorter Xlll wavelengths finally 
leads to the chromatic grating constant tuning at 
A.111 = 513 nm RED, and thus to the new RGB equilibrium 
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[lacuna] 32% R, 35% G and 33% B. 
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The resonant adaptation operation also leads, 
of course, to a geometrical displacement of the 
position of the diffraction orders in the diffraction 
pattern, and thus with reference to the photoelectric 
receivers. However, this displacement always remains 
within the compass of the extent of the receiver 
surfaces . 


